Home audio video display device (AVDD) as sensor monitor

ABSTRACT

Method, computer storage device, and system that establish communication with at least first and second sensors, receive information from the sensors, and determine whether the information from the first sensor satisfies a first alert threshold. It is also determined whether the information from the second sensor satisfies a second alert threshold. Responsive to a determination that the information from the first sensor satisfies the first alert threshold and that the information from the second sensor does not satisfy the second alert threshold, an alert signal is not generated. On the other hand, responsive to a determination that the information from the first sensor satisfies the first alert threshold and that the information from the second sensor satisfies the second alert threshold, the alert signal is generated for output thereof to a display device.

FIELD OF THE INVENTION

The present application relates generally to the use of home audio videodisplay devices (AVDD) such as TVs as sensor monitors.

BACKGROUND OF THE INVENTION

Current sensors, such as household environmental sensors, light sensors,and motion sensors, typically have individual displays associated witheach sensor to display data from each sensor. Present principlesrecognize that the displays are often small and/or low-quality, makingthe displays difficult to read, understand, and ascertain usefulinformation from, among other things.

Also understood herein, the displays are typically positioned in closeproximity to its respective sensor such that multiple displaysassociated with different sensors are often not located in the samegeneral location of e.g., a personal residence. Thus, a need has arisento aggregate the information produced by one or more sensors in a singlelocation for convenient viewing, rather than requiring multiple displaysscattered in different locations to the display data and/or informationand thereby making monitoring of the data and/or information from thesensors burdensome.

SUMMARY OF THE INVENTION

An audio video display device (AVDD) system includes a display and aprocessor controlling the display. The AVDD also includes a computerreadable storage medium accessible to the processor and programmed withinstructions that cause the processor to establish communication with atleast one sensor. The instructions also cause the processor to receiveinformation from the sensor conforming to an application programminginterface (API) provided by a manufacturer of the AVDD to an entityaffiliated with the sensor, or sent from the AVDD to the sensor and thenpresent the information from the sensor on the display in accordancewith the API.

If desired, the API can define, relative to the information from thesensor, content in the information from the sensor to be presented onthe display, where the content is to be presented on the display, andwhen the content is to be presented on the display. As indicated above,in some embodiments the API may be provided by a manufacturer of theAVDD to an entity affiliated with the sensor, while in other embodimentsAPI may be sent from the AVDD to the sensor. Even further, in someembodiments the API is sent from the AVDD to the sensor only in responseto a viewer-input command.

In accordance with present principles, the sensors may be selected,though not required to be exclusively selected, from a group of sensorsconsisting of environmental sensors, ambient light sensors, doorposition sensors, window covering position sensors, pool heaterenergization sensors, motion sensors, and valve position sensors. Thoughpresent principles are described in reference to a personal residentialenvironment, it is to be understood that the same principles may beapplied to sensors and monitoring equipment in. e.g., a hospital or apublic security environment as well.

If desired, the content can include alpha-numeric information only, anicon only, or both an icon and alpha-numeric information. Moreover, insome embodiments the system also includes the sensor. The sensorincludes a processor accessing the API and sending the information tothe AVDD in accordance with the API.

In another aspect, a method includes establishing communication betweenan audio video display device (AVDD) including a display and at leastone sensor. The method then includes receiving information from thesensor(s) conforming to an application programming interface (API)provided by a manufacturer of the AVDD to an entity affiliated with thesensor(s), or sent from the AVDD to the sensor(s). According to themethod, the information from the sensor(s) is then presented on thedisplay in accordance with the API.

In yet another aspect, an audio video display device includes a displayand a processor controlling the display. The device also includes acomputer readable storage medium accessible to the processor andprogrammed with instructions that cause the processor to establishcommunication with at least one sensor. The instructions then cause theprocessor to receive information from the sensor(s) conforming to asoftware interface understandable by the AVDD. Thereafter, theinstructions cause the processor to present the information from thesensor(s) on the display in accordance with the interface.

In other aspects, multiple sensors are monitored simultaneously todetermine if an event has occurred (e.g., a combination of medicalsensors; or home automation sensors indicating sound, movement, etc.,that suggests a break in). A system according to present principles mayre-send the sensor information to the cloud, to either facilitateprocessing, or to be made available to a third party for monitoring thesensor data (parents concerned about a child's security, home securitycompany, medical professionals, etc.).

The details of the present invention, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example system, showing a home AVDDcommunicating with several example sensors for presenting informationfrom the sensors on the AVDD;

FIG. 2 is a flow chart of example logic the AVDD can execute;

FIG. 3 is a schematic diagram showing the data structure of an exampleapplication programming interface (API) that may be pushed by the AVDDto the various sensors automatically when the viewer selects to connectto discovered sensors, or that may be published by a manufacturer of theAVDD to sensor manufacturers so that the sensor manufacturers maypre-program their sensors with the API;

FIGS. 4-10 illustrate example screen shots from the AVDD presentingexample information from various sensors shown in FIG. 1;

FIG. 11 is a block diagram of another example system; and

FIG. 12 is a flow chart of alternate sensor-driven discovery logic.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring initially to FIG. 1, a block diagram of an example systemincluding an audio video display device (AVDD) communicating withseveral example sensors for presenting information from the sensors onthe AVDD is shown. It is to be understood that an AVDD in accordancewith present principles may be a home AVDD such as, but not limited to,a TV. In some embodiments the TV may further be an Internet TV.Notwithstanding the foregoing, it is to be further understood that stillother audio-video display devices may be used in accordance with presentprinciples, such as smart phones, tablet computers, notebook computers,other types of computers, etc.

Thus, a non-limiting system 10 includes an audio video display device(AVDD) 12. The AVDD 12 includes a TV tuner 14 that receives TVprogramming and/or data and/or content for presentation on the AVDD 12.Further, the TV tuner 14 communicates with a processor 16 accessing atangible computer readable storage medium 18 such as, but not limitedto, disk-based or solid state storage. It is to be understood that theprocessor 16 can execute logic in accordance with present principles.The AVDD 12 may also include a non-limiting TV signal input 20 allowingthe AVDD 12 to connect to, e.g., a television head end, cablecommunication link, or satellite communication link for receiving TVprogramming and/or data or content for presentation on the AVDD 12.Additionally, the AVDD 12 can output audio on one or more speakers 22.

Continuing in reference to FIG. 1, it is to be understood that the AVDD12 can connect to the Internet using an internet interface 23 such asbuilt-in wired or wireless modem that communicates with the processor 16to, e.g., send and receive data over the internet “I” or receivestreaming video. Without limitation, the Internet interface may be aWi-Fi interface. Wireless telephony interfaces may also be used. TheInternet “I” includes computers, typically servers, and data storage,along with communication switches.

Regardless of the source of the content, video is presented undercontrol of the processor 16 on a display 24, such as a high definitionTV (HDTV) flat panel display. In some embodiments, the display 24 may bea touch screen display. Also, user commands to the processor 16 may bewirelessly received from a remote commander (RC) 26 using, e.g., RF orinfrared.

The AVDD 12 shown in FIG. 1 also has a sensor communication interface 30that communicates with the processor 16 to execute the functions andlogic in accordance with present principles, among other things. It isto be understood that the sensor communication interface 30 can alsoestablish communication with one or more communication interfaces ofrespective sensors in accordance with present principles. FIG. 1therefore shows various exemplary sensors with respective communicationinterfaces for communicating with the sensor communication interface 30of the AVDD 12.

Thus, an irrigation valve position sensor assembly 32 includes acommunication interface 34 that can communicate with the sensorcommunication interface 30 to send information sensed or gathered or thelike by a valve position sensor 36. Note that while the interfaces 30,34 are shown as separate components in addition to the Internetinterface 23, it is to be understood that the communication interfacesbetween the AVDD 12 and the sensors may be via Internet interfaces suchas Wi-Fi interfaces. Or, the interfaces may be short range interfacessuch as sonic interfaces, IR interfaces, or Bluetooth interfaces. Thus,information from the sensors herein may be sent to the “cloud” (e.g.,Internet “I”) to facilitate processing of the sensor data and then sentback to the AVDD, and/or to facilitate making the data available to athird party for monitoring of the data and providing output signals andalarms to, e.g., wireless computing devices of parents for home/childsecurity, medical professionals, etc.

The information pertains to the operation and/or status of one or moreirrigation valves 38. The valve position sensor assembly 32 alsoincludes a processor 40 communicating with the communication interface34. In non-limiting embodiments, the processor 40 may cause the valveposition sensor 36 to sense and/or gather information regarding theoperation of the irrigation valves 38, and/or may receive informationfrom the valve position sensor 36 regarding the operation of theirrigation valves 38. If desired, the processor 40 may store theinformation from the valve position sensor 36 on a storage medium 42.Regardless of whether the information is stored on the storage medium42, the processor 40, being in communication with the communicationinterface 34, can provide the information from the valve position sensor36 to the communication interface 34 and cause the communicationinterface 34 to send the information to the sensor communicationinterface 30 of the AVDD 12 so that the AVDD 12 can present theinformation.

FIG. 1 also shows a window covering position sensor assembly 44 thatincludes a window covering position sensor 46 for sensing the positionand/or movement of a window covering, a pool heater sensor assembly 48that includes a pool heater sensor 50 for sensing the status and/oroperation of a pool heater and even the temperature of a pool, and adoor position sensor assembly 52 that includes a door position sensor 54for sensing information pertaining to the operation and/or status of adoor 60 such as a residential garage door. Also shown in FIG. 1 is anenvironment sensor assembly 62 that includes an environment sensor 64for sensing, e.g., the household temperature and humidity of theresidence in which the AVDD 12 is disposed. FIG. 1 also includes amotion sensor assembly 66 that includes a motion sensor 68 for sensingmotion and a light sensor assembly 70 that includes a light sensor 72for sensing light.

It is to be understood that the assemblies 44, 48, 52, 62, 66, and 70include respective processors 74, 76, 78, 80, 82, and 84 for causingeach assembly's respective sensor to sense, gather, and/or receiveinformation from the respective sensor in accordance with presentprinciples. The respective processors 74, 76, 78, 80, 82, and 84 alsocommunicate with respective sensor communication interfaces 98, 100,102, 104, 106, and 108 in accordance with present principles to sendinformation sensed, gathered, and/or received or the like from eachassembly's respective sensor to the sensor communication interface 30 ofthe AVDD 12 for presentation thereon. It is to be understood that thecommunication interfaces referenced herein, including the interfaces 30,34, 98, 100, 102, 104, 106, and 108 may support and/or include auniversal serial bus (USB) connection, wired TCP/IP, WiFi TCP/IP, and/orbuilt-in RF transceivers (such as ZWAVE, ZigBee, etc.) in non-limitingembodiments. Furthermore, if desired, the respective processors 74, 76,78, 80, 82, and 84 of the assemblies 44, 48, 52, 62, 66, and 70 maystore the information from the respective sensors on respective storagemediums 86, 88, 90, 92, 94, and 86.

One or more biometric sensors 109 may also be provided and may sendsignals representing biometric parameters of a person to the AVDD. Abiometric sensor typically may include a processor “P”, computer memory“M”, and communication interface “CI” according to principles above.Non-limiting examples of biometric sensors include heart rate sensors,temperature sensors, blood pressure sensors, blood sugar sensors, andthe like.

Moving now to FIG. 2, a flow chart of example logic an AVDD, such as theAVDD 12, executes in accordance with present principles is shown.Beginning with block 110, in an example embodiment one or more sensorssuch as the irrigation valve sensor or the pool heater sensor referencedabove can be discovered through device discovery principles known in theart, although in other embodiments a user can enter sensor communicationinformation into the AVDD to establish communications manually. E.g.,the sensors may be discovered based on wireless signals emitted by asensor within a particular radius of a given location, such as apersonal residence. Moving from block 110 to block 112, the logicprompts a user of the AVDD to decide to connect to the one or moresensors that were discovered at block 110. If the user provides input,e.g. via a remote commander such as the RC 26 described above, inresponse to the prompt commanding the AVDD to connect to the discoveredsensors, the logic continues to block 114.

At block 114, the logic “pushes” and/or provides an applicationprogramming interface (API) from the AVDD to the sensor(s). However, itis to be understood that in some embodiments, the API may be provided bya manufacturer of the AVDD to an entity affiliated with the sensor(s) inaccordance with present principles, rather than having the AVDD push theAPI to the sensors. In such an embodiment, the sensor(s) and/or theirprocessors are already capable of providing information in the desiredAPI when the sensor is discovered back at block 110. Thus, innon-limiting embodiments an entity affiliated with the sensor(s) may be,e.g., a sensor manufacturer vending sensor(s) already being able toprovide information in the appropriate API. In other instances, e.g., anentity affiliated with the sensor may be a third party such as a sensortechnician that provides and/or pushes the API to the sensor wheninstalling the sensor at a particular location (and thus prior to beingdiscovered by the AVDD in accordance with present principles).

Continuing in reference to FIG. 2, at block 116 the logic receivesinformation back from the sensor(s). Concluding FIG. 2 at block 118, theinformation from the sensor(s) is displayed on the AVDD in accordancewith the API.

Thus, it is to be understood that one or more sensors such as but notlimited to the sensors described herein include respective processorsaccessing the API and sending the information to an AVDD such as theAVDD 12 in accordance with the API. To reiterate, sensors in accordancewith present principles may include, but are not limited to,environmental sensors, ambient light sensors (such as photodiodes inmonitored rooms or areas), door position sensors, window coveringposition sensors, pool heater energization sensors, motion sensors (suchas the motion sensor(s) described in U.S. Pat. No. 7,755,052,incorporated herein by reference), valve position sensors, biometricsensors, and/or other sensors including simple switch sensors.

Now referring to FIG. 3, a schematic diagram showing a data structure ofan exemplary API that may be pushed and/or provided by the AVDD tovarious sensors automatically when the viewer selects to connect todiscovered sensors, or that may be published by a manufacturer of theAVDD to sensor manufacturers so that the sensor manufacturers maypre-program their sensors with the API, is shown. It is to be understoodthat the API defines, relative to the information from the sensor,content in the information from the sensor to be presented on a displayof an AVDD, such as the display 24 referenced above. In some embodimentsthe API also defines when and where the content is to be presented onthe display.

Thus, as may be appreciated from FIG. 3, a data structure 120 includesvarious parameters, such as whether an icon from a sensor should bepresented, whether an icon from an AVDD such as the AVDD 12 should bepresented, and/or whether alpha-numeric information from the sensorshould be presented. Accordingly, content parameters 122 can includetypes of content to be presented, such as one or more icons and/oralpha-numeric information. Furthermore, in some embodiments the contentmay include, e.g., only one icon or only alpha-numeric information. Asmay also be appreciated from FIG. 3, the data structure 120 may alsoinclude parameters 124 regarding where and/or how content should bepresented. For example, content may be presented in a full-screen modesuch that only the content is displayed, or in, e.g., a bottom portionof the display or a top portion of the display such that the content maybe simultaneously displayed with unrelated content such as a televisionprogram or motion picture.

In addition to the above, the data structure 120 may also includeparameters 126 regarding when the content from the sensors should bepresented. For example, content may be presented only upon receipt ofthe content or information, and/or receipt of a message containing thecontent and/or information. Accordingly, in non-limiting embodimentsextensible markup language (XML) messaging may be used such that, e.g.,the XML is used to encapsulate the content, information, and/or message.Alternatively or in addition to the above, content may be presented atpredetermined intervals, such as, e.g., every 5 minutes or every hour.Thus, FIG. 3 shows that the data structure 120 includes a parameter forboth presenting content upon receipt and at least one reminder, such asfive minutes after the content is received. If desired, reminders canalso be repeatedly presented at predetermined intervals. Furthermore, insome embodiments audio alerts and/or audible content pertaining toinformation and/or content received from the sensor(s) may be presentedon the AVDD through, e.g., the speakers 20, in lieu of or in addition topresentation of visual content.

Now in reference to FIGS. 4-10, illustrative exemplary screen shots froman AVDD presenting information and/or content from various sensors suchas those referenced above are shown. Thus, FIG. 4 shows that live videomay be displayed along with a visual indication on the bottom portion ofa display, such as the display 24 referenced above, that an irrigationsystem such as lawn sprinklers is on, being based on informationreceived from a valve position sensor. FIG. 5 shows that live video maybe displayed along with a visual indication on the bottom portion of thedisplay that window blinds are opening, being based on informationreceived from a window coverings position sensor. FIG. 6 shows that livevideo may be displayed along with a visual indication on the bottomportion of the display that a pool heater is on, being based oninformation received from a pool heater sensor. FIG. 7 shows that livevideo may be displayed along with a visual indication on the bottomportion of the display of non-limiting environmental statistics such astemperature and humidity levels, being based on information receivedfrom an environment sensor. FIG. 8 shows that a visual indication may bepresented on the display indicating, e.g., an “intruder alert” based onthe motion of a person sensed by a motion sensor.

Continuing in reference to the exemplary screen shots disclosed herein.FIG. 9 shows that live video may be displayed along with a visualindication on the bottom portion of the display that the garage door ofthe residence where the AVDD is disposed is opening, being based oninformation received from a door position sensor. Concluding with FIG.10, live video may be displayed along with a visual indication includingan icon on the bottom portion of the display that a light has beenturned on in another room of a personal residence in which the AVDD isdisposed, being based on information received from a light sensor.

Additionally, note that a user of an AVDD such as the AVDD 12 may choosewhether or not to display information and/or content from the sensors inaccordance with present principles. Thus, for example, a user interfacemay be presented on the AVDD allowing a user to enable presentation ofthe information and/or content from one or more sensors, or to disablepresentation of the information and/or content. Furthermore, it is to beunderstood that one AVDD presenting information and/or content may, e.g.using Internet capabilities, forward the information and/or content toother AVDDs for presentation thereon. Thus, for example, an AVDD such asa TV may present a user interface to a user allowing the user to forwardinformation and/or content from one or more sensors from the TV to theuser's laptop computer or smart phone so that the information and/orcontent may still be monitored by a user when not viewing the TV.

FIG. 11 shows an alternate system in which an AVDD 12 a may communicatewith one or more sensors 32 a along a direct communication path, whichportion of the system can be identical in operation and configuration tothe system shown in FIG. 1. In addition, a home automation platform(HAP) 150 such as, for example, a computer with processor, computerreadable medium, display, input device, etc. may receive informationfrom the sensor 32 a along an aggregate path and provide aggregatedsensor information to the AVDD 12 a as shown. Thus, when the sensor 32 ais a power sensor such as a current sensor, information from it can beaggregated over a period of time, e.g., 24 hours, by the HAP 150 andthen the aggregated sensor information, in this example, total powerusage for the past 24 hours, can be presented on the AVDD 12 a alongwith current power use provided from the sensor 32 along the direct pathas shown.

Furthermore, the HAP 150 (or a sensor or the AVDD or a cloud server) canaggregate data from multiple sensors and provide that aggregated data tothe AVDD 12 a. For example, the HAP 150 may provide to the AVDD 12 a fordisplay data indicating that a thermostat has reached a threshold and isactivating a climate control unit such as a heater or air conditioner inresponse, as indicated by a climate control sensor, that the fanassociated with the unit is at a particular speed as indicated by a fanspeed sensor, and that the current room temperature as sensed by atemperature sensor is at a particular value. In essence, the HAP 150correlates input from different but related sensors and provides thatinput to the AVDD for convenient simultaneous presentation of thevarious inputs from the different but related sensors. Furthermore, theprovision of the HAP 150 facilitates the AVDD working with legacysensors that may not have the capability to execute the API discussedabove but that can communicate with the HAP 150 using legacy protocolsdifferent from the above-discussed API, with the HAP 150 thencommunicating with the AVDD using the above-discussed API.

In some examples, one or more of the devices above (e.g., the HAP orAVDD) monitor plural sensors as described and output an event only iftwo or more sensor outputs meet “event” criteria. For example, a soundsensor and a motion sensor may both output signals representing thepresence of moving humans and animals, and an event “intruder alert” maybe generated only responsive to both sensor signals indicating anintruder, i.e., only in response to the sound sensor outputting a signalindicating noise or human speech above (or below) a threshold and themotion sensor indicating a moving object. Responsive to only one sensoroutputting a signal indicating an intruder, the “intruder alert” may notbe presented, e.g., on the AVDD, in audible and/or visible form. Theintruder alert may be sent to the cloud (Internet) for provisioning ofthe alert to, e.g., a wireless device registered in the cloud to theowner of the premises in which the sensors are disposed, and/or to lawenforcement agencies.

As a further example, a “medical alert” may be generated by, e.g., theHAP or AVDD responsive to two or more biometric sensors outputtingsignals satisfying a medical alert threshold, but not responsive to onlya single biometric sensor outputting such a signal. For example, a“medical alert” may be output (locally and/or to the cloud) responsiveto two or more biometric sensors outputting signals satisfying “alert”thresholds. Thus, a medical alert may be generated only responsive toboth a heart rate sensor indicating a pulse in excess of a threshold anda temperature in excess of a threshold, but not responsive to only asingle biometric sensor outputting a signal indicating a medicalemergency.

FIG. 12 illustrates that device discovery may be initiated on the sensorside, commencing at block 152. When a sensor discovers the AVDD it mayrequest to be added to the AVDD's sensor group at block 154, and if theAVDD accepts the sensor at decision diamond 156, the AVDD assumescontrol of communication with the sensor at block 158. This may includepresenting an onscreen instruction to the user to take particular actionwith respect to the sensor, e.g., operating a button or key on thesensor in a particular fashion to authenticate the sensor to the AVDD.The AVDD can then accept further communication from the sensor.

On the other hand, if the AVDD does not accept sensor communication atdecision diamond 156 the logic flows to block 160 in which the sensorwaits in standby for future instructions, if any, from the AVDD.

While the particular HOME AUDIO VIDEO DISPLAY DEVICE (AVDD) AS SENSORMONITOR is herein shown and described in detail, it is to be understoodthat the subject matter which is encompassed by the present invention islimited only by the claims.

What is claimed is:
 1. Apparatus, comprising: at least one computermemory that is not a transitory signal and that includes instructionsexecutable by at least one processor to configure the processor to: sendto at least one sensor an application programming interface (API)defining one or more of: (a) content in information received from the atleast one sensor to be presented on a display, (b) where the content isto be presented on the display, (c) when the content is to be presentedon the display; receive information from the at least one t sensorconforming to the API; and present the information from the at least onesensor on the display in accordance with the API.
 2. The apparatus ofclaim 1, wherein the API defines, relative to the information from theat least one sensor, two or more of: content in the information from theat least one sensor to be presented on the display, where the content isto be presented on the display, and when the content is to be presentedon the display.
 3. An audio video device (AVD), comprising: at least oneprocessor; a computer memory accessible to the processor and programmedwith instructions that cause the processor to: send to at least onesensor an application programming interface (API) defining one or moreof (a) content in information received from the at least one sensor tobe presented on a display, (b) where the content is to be presented onthe display, (c) when the content is to be presented on the display;receive information from the at least one sensor conforming to the API;and present the information from the at least one sensor on the displayin accordance with the API.
 4. The AVD of claim 3, wherein the APIdefines, relative to the information from the first sensor, two or moreof content in the information from the first sensor to be presented onthe display, where the content is to be presented on the display, andwhen the content is to be presented on the display.